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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 . 1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months atter the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )□ Responsive to communication(s) filed on 19 August 2002 . 
2a)M This action is FINAL. 2b)n This action is non-final. 

3) n Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under £xpa/te Quayle, 1935 CD. 11, 453 O.G. 213. 
Disposition of Claims 

4) M Claim(s) 1-2,4-11,13-14 and 21-26 is/are pending in the application. 

4a) Of the above claim(s) 3,12 and 15-20 is/are withdrawn from consideration. 

5) 0 Claim(s) is/are allowed. 

QM Claim{s) 1-2, 4-11. 13-14 and 21-26 is/are rejected. 
?)□ Claim(s) is/are objected to. 

8) 0 Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) 0 The specification is objected to by the Examiner. 

10) 0 The drawing(s) filed on is/are: aO accepted or bO objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 

11) 0 The proposed drawing correction filed on is: a)^ approved b)n disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) 0 The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§ 119 and 120 

13) S Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 

a)l3AII b)n Some*c)n None of: 

1 .(3 Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (POT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) 0 Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 119(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) 0 Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 . 
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2) □ Notice of Draflsperson's Patent Drawing Review (PTO-948) 5) □ Notice of Informal Patent Application (PTai 52) 
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DETAILED ACTION 



1 . This action is in response to Applicant's amendments on August 19, 2002 have 
been considered but are deemed moot in view of new ground of rejections below. The 
applicant amended claims 1-2, 4-10 and 13-14, and added new claims 21-26 and 
canceled 3, 12 and 15-20. The pending claims are 1-2, 4-11, 13-14 and 21-26. 



2. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

3. Claims 1-2, 4, 7, and 13-14 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Jain et al (U.S. Patent No. 5,893,095, hereinafter, "Jain"). 

With respect to claim 1, Jain discloses the method of searching multimedia data, 
(image, col. 6, lines 25-43), receiving at least one reference multimedia data (108, 
visual query input, FIG. 1 A) selected by a user (102, FIG. 1 A), wherein the reference 
multimedia data represents a specified multimedia data to be searched, see (FIG. 1A, 



Claim Rejections - 35 USC § 103 



Application/Control Number: 09/495,250 Page 3 

Art Unit: 2172 

col. 9, lines 10-67 to col. 10, lines 1-10, FIG. 5A, col. 11, lines 53-67 to col. 12, lines 1- 
5); 

measuring the similarities of features (feature vector, FIG. 5A) included in the 
plurality of reference multimedia data, see (FIG. 5A, col. 12, lines 23-60); 

determining and updating weights of each feature (query optimization) according 
to the measured similarities of the features, see (col. 8, lines 35-55, col. 17, lines 1-51, 
col. 19, lines 29-67 to col. 20, lines 1-14); 

searching for the specified multimedia data utilizing features and feature 
elements included in the plurality of reference multimedia data, in consideration of the 
updated weights of features and feature elements, see (FIG.1A, FIG. 11, FIG. 5B, col. 
9, lines 9-26, col. 12, lines 54-67 to col. 13, lines 1-35, col. 22, lines 43-67 to col. 23, 
lines 1-65). Jain discloses measuring the similarities of the feature in each feature 
included in the plurality of reference multimedia data, see (FIG. 5A, col. 12, lines 23-60); 

determining weights of each feature elements in respective features according to 
the measured similarities of the feature, see (col. 8, lines 35-55, col. 17, lines 1-51, col. 
19, lines 29-67 to col. 20, lines 1-14). Jain does not explicitly disclose feature elements. 
However, Jain teaches the primitives are attributes for color, texture, and shape, those 
are called features of images, see (col. 6, lines 25-43, col. 12, lines 23-53). The color is 
one of the primitives of query image and global color is one of the features of query 
image and local color is the feature of elements, see (FIG. 5A, col. 12, lines 6-22). 
Therefore, it would have been obvious a person having ordinary skill in the art to include 
feature element into the system of Jain to specify small feature elements from features. 



Application/Control Number: 09/495,250 Page 4 

Art Unit: 2172 

The image comprises features (color, texture, and shape) globally (i.e., entire image, 
features) and locally (i.e, over small region of image, feature elements) and user can 
give relative importance of each features to get closer target images that user wants. 
Thus, it would be useful to use and specify features and feature element to get exact 
target images from image searching engine. 

With respect to claim 2, Jain discloses that searching for the specified multimedia 
data utilizing a combination of features and feature elements of the respective features 
included in the at least one reference multimedia data, wherein each feature has a 
feature weight, and each feature element has a feature element weight, see (col. 12, 
lines 6-61). 

With respect to claim 4, Jain discloses the terminating the search if the user is 
satisfied with the result of the search, 

receiving at least one other reference multimedia data selected from among the 
resultant images of the search, wherein the at least one other reference multimedia data 
is determined to be similar to the specified multimedia data, see (FIG. 4, col. 1 1 , lines 
35-53); 

measuring the similarities of features included In the plurality of reference 
multimedia data and the at least one other reference multimedia data, see (FIG. 5A, col. 
12, lines 23-60); 
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determining and updating weights of each feature according to the measured 
similarities of the features, see (col. 8, lines 35-55, col. 17, lines 1-51, col. 19, lines 29- 
67 to col. 20, lines 1-14); 

measuring the similarities of the feature elements in each feature included in the 
plurality of reference multimedia data and the at least one other reference multimedia 
data, see (col. 8, lines 35-55, col. 17, lines 1-51, col. 19, lines 29-67 to col. 20, lines 1- 
14); 

determining and updating weights of each feature elements in respective 
features according to the measured similarities of the feature elements, see (col. 8, lines 
35-55, col. 17, lines 1-51, col. 19, lines 29-67 to col. 20, lines 1-14); and 

re-searching for the specified multimedia data utilizing features and feature 
elements included in the plurality of reference multimedia data and in the at least one 
other reference multimedia data, in consideration of the updated features weights and 
feature elements weights, see (FIG.1A, FIG. 11, FIG. 5B, col. 9, lines 9-26, col. 12, lines 
54-67 to col. 13, lines 1-35, col. 22, lines 43-67 to col. 23, lines 1-65). 

With respect to claim 7, Jain discloses that one reference multimedia data is 
selected by the user, and comprising searching for the specified multimedia data 
utilizing features and feature elements of the respective features included in the 
reference multimedia data, wherein each features has a feature weight and each 
feature element has a feature element weight, see (col. 12, lines 6-67 to col. 13, lines 1- 
35). 
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With respect to claim 13, Jian discloses the method of constructing a multimedia 
data, incorporating a feature information including features of an image, see (FIG.1A, 
FIG. 5A, col. 9, lines 10-67 to col. 1-55,col. 11, lines 54-67 to col. 12, lines 1-60); and 

incorporating a weight information including weight information of the features, 
see (FIG.1A, FIG. 5A, col. 9, lines 10-67 to col. 1-55,col. 11, lines 54-67 to col. 12, lines 
1-60). Jain does not explicitly disclose feature elements. However, Jain teaches the 
primitives are attributes for color, texture, and shape, those are called features of 
images, see (col. 6, lines 25-43, col. 12, lines 23-53). The color is one of the primitives 
of query image and global color is one of the features of query image and local color is 
the feature of elements, see (FIG. 5A, col. 12, lines 6-22). Therefore, it would have been 
obvious a person having ordinary skill in the art to include feature element into the 
system of Jain to specify small feature elements from features. The image comprises 
features (color, texture, and shape) globally (i.e., entire image, features) and locally (i.e, 
over small region of image, feature elements) and user can give relative importance of 
each features to get closer target images that user wants. Thus, it would be useful to 
use and specify features and feature element to get exact target images from image 
searching engine. 

With respect to claim 14, Jian discloses that the feature and the feature 
elements are represented by an image characteristic structure, global information which 
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represents a feature of a whole image, see (col. 11, lines 53-67 to col. 12, lines 1-61); 
and 

spatial information which represents a feature of an image region, wherein the 
image characteristic structure further comprises a weight information which represents 
the importance of the global information and the spatial information, see (col. 11, lines 
53-67 to col. 12, lines 1-61). 

4. Claims 5-6, 8-10 and 21-26 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Jain et al (U.S. Patent No. 5,893,095, hereinafter, "Jain") in view of 
Ishimaru (U.S. Patent No. 5,982,931). 

With respect to claim 5, Jain discloses, receiving at least one multimedia data 
selected from among the resultant images of the search, wherein the at least one 
multimedia data is determined to be to the specified multimedia data, see (FIG. 4, col. 
11, lines 36-52); 

measuring the features included in the plurality of reference multimedia data and 
the at least one multimedia data, see (FIG. 5A, col. 12, lines 23-60); 

measuring the feature elements in each feature included in the plurality of 
reference multimedia data and the at least one multimedia data, see (col. 8, lines 35-55, 
col. 17, lines 1-51, col. 19, lines 29-67 to col. 20, lines 1-14); and 

wherein in determining and updating weights of each feature elements in 
respective features according to the measured the features according to the measured 
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the feature elements, see (col. 8, lines 35-55, col. 17, lines 1-51, col. 19, lines 29-67 to 
col. 20, lines 1-14); 

wherein in determining and updating weights of each feature elements in 
respective features according to the measured the feature elements, see (col. 8, lines 
35-55, col. 17, lines 1-51, col. 19, lines 29-67 to col. 20, lines 1-14); and 

wherein in re-searching for the specified multimedia data utilizing features and 
feature elements included in the plurality of reference multimedia data, in the at least 
one other reference multimedia data and in the at least one data, in consideration of the 
updated features weights and feature elements weights, see (FIG.1A, FIG. 11, FIG. 5B, 
col. 9, lines 9-26, col. 12, lines 54-67 to col. 13, lines 1-35, col. 22, lines 43-67 to col. 
23, lines 1-65). Jain discloses the method for image searching with similarity of 
multimedia. Jain does not explicitly disclose the method for image searching with the 
dissimilarity of multimedia. However, Ishimaru discloses that the images are compared 
and analyzed to determine dissimilarities of images (FIG. 2, col. 3, lines 54-67 to col. 4, 
lines 1-67 to col. 5, lines 1-39, abstract). Therefore, it would have been obvious a 
person having ordinary skill in the art to include the method to get dissimilarity of 
multimedia from Ishimaru for the image searching method of Jain. The image searching 
mechanism has many different features to consider when user queries with sample 
image to get exact target image that user want, using similar image features and 
dissimilar image features. Thus, it would be beneficial to use the similarity and 
dissimilarity combined to get exact target images. 
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Clams 6 and 8-10 are rejected for the reasons set fourth hereinabove for claims 
4 and 5. The claim 4 discloses for the method of image re-searching with similarities of 
images and the claim 5 discloses the method of images searching with dissimilarities of 
images. 

With respect to claims 21 and 24, Jain discloses the searching foe a target image 
based on search criteria, (FIG. 1 A), inputting a first image that is similar to the target 
image, see (FIG.1A, FIG. 2, FIG. 5A, FIG. 5B, col. 9, lines 10-67, col. 10, lines 56-67 to 
col. 11, lines 1-15, col. 11, lines 53-667 to col. 12, lines 1-67); 

correlating the first image and the second image to construct the search criteria, 
see (FIG. 4, col. 11, lines 36-53 and col. 12, lines 62-67 to col. 13, lines 1-35, resulted 
target images are reused for second searching). Jain does not explicitly disclose the 
inputting a second image that is dissimilar to the target image. However, Ishimaru 
discloses that dissimilar image is identified, and inputted for searching, see (FIG. 2, col., 
lines 3, lines 53-67 to col. 4, lines 1-65). Therefore, it would have been obvious a 
person having ordinary skill in the art to include the method of inputting the second 
image that is dissimilar to the target image from Ishimaru into Jain to get the target 
image. The image searching mechanism has many different features to consider when 
user queries with sample image to get exact target image that user want, using similar 
image features and dissimilar image features. Thus, it would be beneficial to use the 
similarity and dissimilarity combined to get exact target images. 
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Claims 22-23 and 25-26 are rejected for the reasons set fourth hereinabove for 
claims 1-2, 4 and 5. 

5. Claim 1 1 is rejected under 35 U.S.C. 103(a) as being unpatentable over Jain et 
al (U.S. Patent No. 5,893,095, hereinafter, "Jain") in view of Taniguchi et a! (U.S. 
Patent No. 5,619,347, hereinafter, "Taniguchi"). 

With respect to claim 1 1 , Jain discloses that the feature is color and the feature 
element weights are determined, see (FIG. 5A, col. 11, lines 53-67 to col. 12, lines 1- 
61). Jain does not explicitly disclose feature elements weights are determined either by 
a color histogram with n color elements as the feature elements, or by dividing a 
multimedia data into n*m grid regions and utilizing a regional color histogram or a color 
representing a grid region as the feature elements. However, Taniguchi discloses that 
the color weights are determined by histogram with n*m elements as the feature 
elements, see (FIG. 2, col. 10, lines 61-67 to col. 11, lines 1-46, FIG. 5, col. 23, lines 24- 
67 to col. 26, lines 1-56, col. 44, lines 14-60). Therefore, it would have been obvious a 
person having ordinary skill in the art to include that the method of the feature elements 
weights are determined either by a color histogram with n color elements as the feature 
elements, or by dividing a multimedia data into n*m grid regions and utilizing a regional 
color histogram or a color representing a grid region as the feature elements from 
Taniguchi into Jain to get the weights of feature elements of color from histogram. The 
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histogram is used to get the balances and weights of color features, which is beneficial 
to get specific color element's weight for specific image searching. 

Conclusion 

6. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

Barber et al (U.S. Patent No. 5,751 ,286) discloses the system for image query 
based upon images features such as colors, texture and shape. 

Rising, III (U.S. Patent No. 6,445,834) discloses the system for image query and 
storage based on a plurality of image features. 

7. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 

§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
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the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Isaac M Woo whose telephone number is (703) 305- 
0081 . The examiner can normally be reached on 8:00-5:00. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Kim Y Vu can be reached on (703) 305-4393. The fax phone numbers for 
the organization where this application or proceeding is assigned are (703) 746-7239 for 
regular communications and (703) 308-6606 for After Final communications. 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the receptionist whose telephone number is (703) 305- 
3900. 



IMW 

October 31, 2002 




SUPERVISORY PATENT EXAMINER 
TECKM0L06Y CENTER 2100 



